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TITLE OF THE INVENTION 

Defect Inspection Apparatus, Defect Inspection Method and Program 



BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to a technique for inspecting pattern on an object. 



Description of the Background Art 

In the field of appearance inspection of pattern formed on a semiconductor 

10 substrate, a color filter, a shadow mask, a high-definition printed circuit board or the like, 
pattern of lead frame, pattern of photomask used for forming these patterns or the like, 
conventionally, a comparison check method has been mainly performed with multitone 
images. For example, a differential absolute value image which indicates absolute 
values of the difference in pixel between an inspection image (an image to be inspected) 

15 and a reference image is obtained and a region in the differential absolute value image 
which has pixel values larger than a predetermined threshold value is detected as a defect. 

In the comparison check method, since it is premised that conditions such as 
dynamic range and brightness of the inspection image and the reference image are equal, 
in order to remove an effect of variation in dynamic range, brightness and the like, also 

20 performed is a method of detecting defects after normalizing a pixel value distribution by 
linear transformation so that pixel values of an image should be appropriately distributed 
in a predetermined range. 

As disclosed in Japanese Patent Application Laid Open Gazette No. 2002-22421, 
proposed is a method of removing an effect of variation in sharpness of image (in other 

25 words, variation in graininess) by calculating a standard deviation of signed difference in 
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pixel value between the inspection image and the reference image and normalizing a 
histogram of the differential absolute value image on the basis of the standard deviation. 

In some cases, the pixel value becomes as large as that in a defective portion in 
some region of the differential absolute value image due to an effect of a surface state of 
5 an object to be picked up or an image pickup part (such as a charge-up phenomenon 
which is found in an image picked up by using an electron beam like an SEM) and the 
like. An unnecessary increase in pixel value of the differential absolute value image is 
caused, in many cases, in an extremely local region consisting of several pixels, and in 
such a case, it is impossible to appropriately detect a defect by correction of pixel value in 

10 a conventional macroscopic method. 

In a case where, for example, dynamic ranges of the inspection image and the 
reference image are matched with each other in a region consisting of about 100X 100 
pixels, an effect of variation in pixel value produced entirely on this region can be 
removed but the variation in pixel value caused by the charge-up phenomenon and the 

1 5 like in the region consisting of several pixels is still incorrectly detected as a defect. 

Fig. 1 is a view showing incorrect detection of a defect due to the charge-up 
phenomenon. In Fig. 1, assuming that ideal values of pixel in a line 911 of an inspection 
image 91 (e.g., values of an reference image) are shown in graph 921 and actual pixel 
values are shown in graph 922, pixel values of the differential absolute value image are 

20 shown in graph 923. 

As indicated by reference sign 922a in graph 922, when the pixel value is 
deviated from the ideal value due to a defect, the pixel value of the differential absolute 
value image becomes larger as indicated by reference sign 923a in graph 923, and this 
allows detection of the defect. When values of several pixels are deviated from the ideal 

25 value as indicated by reference sign 922b in graph 922, however, a defect is incorrectly 
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detected as indicated by reference sign 923b in graph 923. 
can not be avoided by the conventional method. 



Such a local wrong detection 



SUMMARY OF THE INVENTION 
5 It is an object of the present invention to provide a technique for more 

appropriately detecting a defect on an object. 

The present invention is intended for a defect inspection apparatus for inspecting 
pattern on an object. The defect inspection apparatus comprises an image pickup device 
for performing an image pickup of an object to acquire data of an inspection image which 

10 is multitone; a memory for storing data of a reference image; and an operation part for 
obtaining transfer characteristics to enhance difference between arbitrary pixels among a 
plurality of specified pixel values which are specified in defect detection and then 
obtaining an enhanced differential image between the inspection image and the reference 
image on the basis of the transfer characteristics, to perform inspection on the basis of the 

1 5 enhanced differential image. 

In this present invention, by using the enhanced differential image, it is possible 
to perform appropriate inspection with attention to the specified pixel values. Especially, 
by setting a plurality of specified pixel values in accordance with the kind of defect to be 
detected, the inspection can be performed with high precision. 

20 According to one preferred embodiment of the present invention, the transfer 

characteristics is set on the basis of pixel values of the inspection image or the reference 
image, and the operation part converts the inspection image and the reference image on 
the basis of the transfer characteristics to obtain a differential image between a converted 
inspection image and a converted reference image as the enhanced differential image. 

25 The plurality of specified pixel values may be positioned outside a pixel value 



range corresponding to a specific region in the inspection image or the reference image. 
This makes it possible to detect a defect that the pixel value becomes extremely large or 
small. 

The transfer characteristics may include inspection image transfer characteristics 
obtained from the inspection image and reference image transfer characteristics obtained 
from the reference image. In other words, the transfer characteristics may be obtained 
individually as the inspection image transfer characteristics and the reference image 
transfer characteristics or obtained as a synthesis of the inspection image transfer 
characteristics and the reference image transfer characteristics. 

The operation part may synthesize a differential image between the inspection 
image and the reference image and the enhanced differential image and compare values of 
pixels in a synthesized image with a predetermined threshold value, to perform inspection. 
This makes it possible to relieve the degree of enhancement. 

These and other objects, features, aspects and advantages of the present 
invention will become more apparent from the following detailed description of the 
present invention when taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a view showing incorrect detection of a defect; 

Fig. 2 is a diagram showing a construction of an inspection apparatus; 

Fig. 3 is a block diagram showing a structure of a computer; 

Fig. 4 is a diagram showing a functional structure of the computer; 

Fig. 5 is a flowchart showing an operation flow of the inspection apparatus; 

Figs. 6 to 8 are graphs illustrating histograms and transfer curves; 

Fig. 9 is a diagram showing another exemplary functional structure of the 



computer; 

Fig. 10 is a flowchart showing another operation flow of the inspection 
apparatus; 

Figs. 11 and 12 are graphs illustrating histograms and transfer curves; 
Fig. 13 is a view illustrating a reference image; 
Fig. 14 is a graph illustrating a histogram and a transfer curve; 
Fig. 15 is a diagram showing another exemplary functional structure of the 
computer; 

Fig. 16 is a flowchart showing an operation flow of the inspection apparatus; 

Fig. 17A is a graph illustrating a transfer curve; 

Fig. 17B is a view showing a 2-D (two-dimensional) LUT; 

Fig. 18A is a graph illustrating a transfer curve without conversion (i.e., straight 

line); 

Fig. 18B is a view showing a 2-D LUT without enhancement of differential 

image; 

Fig. 19 is a diagram showing another exemplary functional structure of the 
computer; 

Fig. 20A is a graph illustrating transfer curves; 
Fig. 20B is a view showing a 2-D LUT; 

Fig. 21 is a diagram showing another exemplary functional structure of the 
computer; 

Fig. 22 is a flowchart showing part of an operation flow of the inspection 
apparatus; 

Fig. 23 is a diagram showing another exemplary functional structure of the 
computer; 
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Fig. 24 is a view showing a 2-D LUT; 

Figs. 25 A to 25D are graphs showing characteristics of the 2-D LUT; 

Fig. 26 is a flowchart showing an operation flow of the inspection apparatus; and 

Fig. 27 is a view illustrating partial images. 

5 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 2 is a view showing a construction of an inspection apparatus 1 in 
accordance with the preferred embodiment of the present invention. The inspection 
apparatus 1 is an apparatus for inspecting pattern on a semiconductor substrate 

10 (hereinafter, referred to as "substrate") 9 and comprises an image pickup part 2 for 
performing an image pickup of a predetermined region on the substrate 9 to acquire data 
of a multitone object image, a stage 3 for holding the substrate 9 and a stage driving part 
3 1 for moving the stage 3 relatively to the image pickup part 2. 

The image pickup part 2 comprises a lighting part 21 for emitting an illumination 

15 light, an optical system 22 for guiding the illumination light to the substrate 9 and 
receiving the light from the substrate 9 and an image pickup device 23 for converting an 
image of the substrate 9 formed by the optical system 22 into an electrical signal. This 
image pickup part 2 may be an image pick-up device using an electron beam. The stage 
driving part 31 has an X-direction moving mechanism 32 for moving the stage 3 in the X 

20 direction of Fig. 2 and a Y-direction moving mechanism 33 for moving the stage 3 in the 
Y direction. The X-direction moving mechanism 32 has a construction in which a ball 
screw (not shown) is connected to a motor 321 and moving the Y-direction moving 
mechanism 33 in the X direction of Fig. 2 along guide rails 322 with rotation of the motor 
321. The Y-direction moving mechanism 33 has the same construction as the 

25 X-direction moving mechanism 32 and moves the stage 3 in the Y direction along guide 
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rails 332 by its ball screw (not shown) with rotation of its motor 331. 

The inspection apparatus 1 further comprises a computer 5 constituted of a CPU 
which performs various computations, memories which store various information and the 
like. The computer 5 performs a detection of defect and also serves as a control part 
5 which controls other constituent elements in the inspection apparatus 1 . 

Fig. 3 is a block diagram showing a structure of the computer 5. The computer 
5 has a constitution of general computer system where a CPU 5 1 for performing various 
computations, a ROM 52 for storing a basic program and a RAM 53 for storing various 
information are connected to a bus line. To the bus line, a fixed disk 54 for storing 
10 information, a display 55 for displaying various information such as images, a keyboard 
56a and a mouse 56b for receiving an input from a user, a reader 57 for reading 
information from a computer-readable recording medium 8 such as an optical disk, a 
magnetic disk or a magneto-optic disk, and a communication part 58 for transmitting and 
receiving a signal to/from other constituent elements in the inspection apparatus 1 are 
1 5 further connected through an interface (I/F) as appropriate. 

A program 80 is read out from the recording medium 8 through the reader 57 
into the computer 5 and stored into the fixed disk 54 in advance. The program 80 is 
copied to the RAM 53 and the CPU 51 executes computation in accordance with the 
program stored in the RAM 53 (in other words, the computer executes the program), and 
20 the computer 5 thereby performs an operation of defect detection. 

Fig. 4 is a diagram showing a functional structure of the computer 5. In Fig. 4, 
an input part 56 corresponds to the keyboard 56a or the mouse 56b of Fig. 3 and an image 
memory 531 and a memory 532 correspond to the RAM 53 of Fig. 3. The image 
memory 531 may be a dedicated memory which is additionally provided in the computer 
25 5. 
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A range setting part 501, an LUT (lookup table) generation part 502, an image 
conversion part 503, a subtraction part 504, a comparison part 505 and a display control 
part 506 represent functions performed by the CPU 51, the RAM 53 and other peripheral 
circuits when the CPU 51 performs a computation in accordance with the program 80. 
5 An operation of the inspection apparatus 1 will be discussed below, referring to Figs. 2 to 
4. 

Fig. 5 is a flowchart showing an operation flow of the inspection apparatus 1 . 
In the inspection apparatus 1, first, the computer 5 controls the stage driving part 31 to 
relatively move an image pickup position corresponding to the image pickup part 2 to a 

10 predetermined position over the substrate 9 and the image pickup part 2 acquires data of 
object image of multitone (e.g., 256 tones if 8 bits). The image pickup (i.e., prescan) is 
performed for a region which is supposed to have no defect and the acquired data is 
stored in the image memory 531 and prepared as reference image data 601 (Step SI). 

Next, the range setting part 501 sets a range of values of specified pixels in 

15 defect detection (hereinafter, referred to as a "specified pixel value range") (Step SI 2). 
The technical meaning of the specified pixel value range will be discussed later. It is 
selected in advance whether the setting of specified pixel value range is performed 
manually by a user or automatically. 

When the setting of specified pixel value range is performed manually, first, the 

20 range setting part 501 acquires the reference image data 601, generates a histogram of 
pixel values of reference image and displays the histogram on the display 55. Fig. 6 is a 
graph illustrating a histogram 62a. When a histogram has two maximum values (this is 
hereinafter referred to as "bi-modality"), like the histogram 62a, a range near the center 
minimum value is set by the user as a specified pixel value range 63. The setting is 

25 performed by the range setting part 501 when the input part 56 receives the operation of 
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the user. When the reference image has two kinds of regions (for example, a wiring 
pattern region and the other background region on the substrate 9), a range between 
respective ranges of pixel values which are supposed to be obtained from these regions is 
set as the specified pixel value range 63. In the manual setting, the specified pixel value 

5 range 63 is set in accordance with the intention of the user. 

When the setting of specified pixel value range is performed automatically, for 
example, an average value around each of the peaks and a standard deviation o of 
distribution around each of the peaks are obtained for the histogram 62a having 
bi-modality as shown in Fig. 6 and a range between ranges 641 and 642 each of which 

10 covers (±o) of the corresponding peak centered at the average value of the peak is set as 
the specified pixel value range 63. The ranges 641 and 642 correspond to ranges of 
pixel values obtained from both the regions when the reference image has two kinds of 
regions. By using the standard deviation, it is possible to appropriately set the pixel 
value ranges corresponding to these regions. 

15 On the other hand, when a histogram does not have bi-modality like a histogram 

62b (in other words, has a small contrast) as shown in Fig. 7, first, a threshold value TH is 
set and average values AVI and AV2 of values of pixels belonging to predetermined 
ranges 651 and 652 on both sides of the threshold value TH are obtained. Then, a range 
between the average value AVI and the average value AV2 is set as the specified pixel 

20 value range 63. When the reference image has two kinds of regions (for example, a 
wiring pattern region and the other background region on the substrate 9), the ranges 651 
and 652 are ranges of pixel values which are supposed to be obtained from these regions. 
In other words, the average values AVI and AV2 correspond to average values of values 
of the pixels belonging to these regions. Intermediate values or the like may be used 

25 instead of the average values AVI and AV2. 
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As a method of setting a threshold value TH1, a method of setting a threshold 
value in binarization of a multivalued image or other various methods may be adopted. 
For example, a method disclosed in "An Automatic Threshold Selection Method Based 
on Discriminant and Least Squares Criteria M by Nobuyuki Otsu (IEICE (The Institute of 
5 Electronics, Information and Communication Engineers) Transactions, '80/4 vol. J63-D, 
No. 4, pp. 349-356) may be used. In this method, as a value for evaluation on propriety 
of a threshold value, measures of class separability based on within-class variance and 
between-class variance (herein, "class" refers to a group of pixel values which are divided 
by the threshold value) are adopted, and a threshold value is obtained so that the measures 

10 of class separability can be the maximum. By this method, even if a histogram of pixel 
values has no bi-modality when an image is divided into two regions, it is possible to 
steadily obtain an optimum threshold value in a non-parametric manner. 

As a method of setting the threshold value TH, well known is Kittler's method 
where ambiguity which indicates which region (any one region among a plurality of 

1 5 regions of the reference image) an observed value (pixel value) belongs to is defined and 
a threshold value is set on the basis of a principle that the ambiguity is made minimum for 
the worst distribution (in other words, the normal distribution). Naturally, other various 
methods may be used as a method of setting the threshold value TH. 

When the specified pixel value range 63 is set, next, the LUT generation part 502 

20 obtains an LUT (lookup table) for converting the image (Step SI 3). Specifically, the 
LUT which corresponds to the transfer curve indicated by reference sign 71 in Figs. 6 and 
7 is obtained. An obtained LUT 61 is stored in the memory of the image conversion part 
503 as shown in Fig. 4. For the transfer curve 71, the horizontal axis and the vertical 
axis correspond to pixel values. Specifically, in the transfer curve 71, a pixel value on 

25 the horizontal axis (e.g., any one value within a range from 0 to 255) is transferred to a 
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pixel value on the vertical axis (this value may be a value within the range from 0 to 255 
or in a broader range with more tones). 

The inclination of the transfer curve 71 is made larger in the specified pixel value 
range 63 than in other pixel value ranges. As a result, by the transfer characteristics in 
5 accordance with the transfer curve 71, the difference between any two pixel values in the 
specified pixel value range 63 is enhanced (in other words, the difference between any 
two pixel values in the specified pixel value range 63 is enlarged relatively to the 
difference between the two pixel values in other pixel value ranges). When one of the 
pixel values is included in the specified pixel value range 63 and the other pixel value is 

10 not included in the specified pixel value range 63, the difference between the two pixel 
values is relatively enhanced to some degrees as compared with the case where both pixel 
values are not included in the specified pixel value range 63. 

The image conversion part 503 transfers the pixel values of the reference image 
by using the LUT 61, generates a converted reference image and stores the converted 

15 reference image in the image memory 531 as converted reference image data 602 (Step 
SI 4). Through the above operations, preparation for inspection of pattern on the 
substrate 9 is completed. 

Next, the inspection apparatus 1 moves a region on the substrate 9 to be 
inspected to a portion immediately below the image pickup part 2 to perform an image 

20 pickup and the image pickup part 2 acquires data of inspection image (i.e., image to be 
inspected) (Step SI 5). On the substrate 9, there are a plurality of blocks in which the 
same patterns are formed, and an image of specific pattern in one block is acquired as the 
reference image and an image of the same pattern (in other words, a region corresponding 
to the reference image) in other blocks is acquired as an inspection image. 

25 The pixel values of the inspection image are subsequently inputted to the image 
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conversion part 503, the value of each pixel of the inspection image is transferred in 
accordance with the LUT 61 and the result is outputted to the subtraction part 504. With 
this operation, data of converted inspection image is substantially obtained (Step SI 6). 

At the same time when the pixel value of the converted inspection image is 
5 inputted to the subtraction part 504, the pixel value corresponding to the converted 
reference image is inputted from the image memory 531 and the subtraction part 504 
obtains a differential absolute value between these pixel values. With this operation, the 
subtraction part 504 substantially obtains the differential absolute value image between 
the converted inspection image and the converted reference image (hereinafter, referred to 

10 as an "enhanced differential image") (Step SI 7). 

The differential absolute value obtained by the subtraction part 504 is compared 
with a predetermined threshold value by the comparison part 505 (Step SI 8), and when 
the differential absolute value is smaller than the threshold value, it is determined that the 
corresponding pixel in the inspection image is not included in a defect and when the 

15 differential absolute value is larger than the threshold value, it is determined that the 
corresponding pixel in the inspection image is included in a defect. A set of comparison 
results corresponding to the whole inspection image is stored in the memory 532 as 
inspection result data 603. The inspection result data 603 is displayed as binary image 
on the display 55 through the display control part 506 if necessary (Step SI 9), and then 

20 the user confirms whether there is a defect or not. 

Thus, in the inspection apparatus 1 , the inspection image and the reference image 
are converted in accordance with the LUT 6 1 and inspection is performed by using the 
differential absolute value image between these images after conversion. Since the LUT 
61 is used for such transfer (or conversion) as to enhance the difference between any two 

25 pixel values in the specified pixel value range, when a value of pixel at a position in the 
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inspection image and a value of pixel corresponding to the reference image are included 
in the specified pixel value range, the difference between these pixel values is enhanced 
(relatively to the case where these pixel values are outside the specified pixel value 
range). 

5 When the reference image has two kinds of regions, e.g., a wiring pattern region 

and the other background region on the substrate 9, since the specified pixel value range 
is positioned at least between representative values of the pixel values corresponding to 
these two regions (the most preferable representative value is an average value which is 
easily obtained) (though the specified pixel value range corresponds to a pixel value range 

10 between the pixel value ranges obtained from these two regions in the case of Fig. 6, this 
pixel value range is positioned between the representative values), the pixel value of the 
enhanced differential image is enhanced by transfer in accordance with the LUT 61, 
relatively to a pixel which is substantially uncertain on which one of the two kinds of 
regions it belongs to (or which has relatively small possibility of belonging to any one of 

1 5 the two regions). 

Through the above operations, for example, by setting the specified pixel value 
range 63 so that the pixel values in the inspection image should not be included even if 
there is variation in pixel value caused by the charge-up phenomenon of the image pickup 
part 2 or the like, it is possible to make the pixel value in the enhanced differential image 

20 which corresponds to the charge-up phenomenon relatively smaller than the pixel value 
corresponding to by a defect. As a result, the difference in pixel value between the 
inspection image and the reference image which is caused by a defect can be enhanced 
and it becomes possible to appropriately detect a defect by comparing the pixel value in 
the enhanced differential image with the predetermined threshold value. 

25 By setting a pixel value range between the representative values in the pixel 
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value ranges obtained from two regions in the reference image as the specified pixel value 
range, the conversion in accordance with the LUT may be regarded as a conversion which 
can reduce variation in pixel value in a region (e,g., the wiring pattern region or the 
background region) relatively (to a pixel value which is uncertain on which region it 
5 belongs to) or increase the difference of respective average values of the pixel values in 
the two regions (in other words, the approximate difference in pixel value between the 
two regions). As a result, it is possible to enhance the difference in pixel value between 
the regions in the enhanced differential image (or decrease of the difference of pixel 
values within one region or decrease of the difference of pixel values within one region 

10 while enhancing the difference in pixel value between the regions). 

As shown in Fig. 7, by obtaining a threshold value for region division in a 
non-parametric manner from the reference image (or from the inspection image as 
discussed later), it is possible to appropriately perform a defect detection even if the 
difference in pixel value between a defect and a non-defect is small (or the contrast of 

1 5 image is small). 

Fig. 8 is a graph illustrating another exemplary specified pixel value range. Fig. 
8 shows a state where a range of pixel values which is larger than the pixel value range in 
which there are frequencies to some degrees is set as the specified pixel value range 63. 
When the reference image has two kinds of regions (for example, a wiring pattern region 

20 and the other background region on the substrate 9), a range of pixel values which is 
larger than each of the pixel value ranges 641 and 642 obtained from these regions is set 
as the specified pixel value range 63. Also in this case, the specified pixel value range 
63 may be set manually by the user or automatically on the basis of the average values or 
the standard deviations of the peaks. 

25 Then, as indicated by reference sign 72, a transfer curve (i.e., the LUT 61) 
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enhancing the difference of any two pixel values in the specified pixel value range 63 is 
obtained. According to the transfer curve 72, when a pixel value which is extremely 
larger than the normal pixel value is found in the inspection image at a certain position, 
the pixel value at this position is enhanced in the enhanced differential image and easily 
5 detected as a defect. For example, in a case where the illumination light is abnormally 
reflected due to a foreign material defect having a specific reflectance on the substrate 9 
and the pixel value in the inspection image becomes larger, it is possible to easily detect a 
defect. 

It is preferable that the range 642 should be set large for the purpose of detecting 
10 a defect that a pixel value becomes extremely large. For example, a range consisting of 
pixel values equal to or more than a specified value obtained by adding three times the 
standard deviation a corresponding to the peak on the right side to an average value near 
the peak may be set as the specified pixel value range 63. By using the standard 
deviation, it is possible to appropriately obtain the pixel value range 642 corresponding to 
1 5 a specific region (such as the wiring pattern region) and position the specified pixel value 
range 63 outside the pixel value range 642 easily (in other words, on the basis of a certain 
criteria). 

Thus, the specified pixel value range is set as a range of pixel values in the 
inspection image (or the reference image) where a defect is to be detected. By setting 
20 the specified pixel value range in accordance with the type of defect to be detected, it is 
possible to achieve detection of various defects with high precision. 

Fig. 9 is a diagram showing another exemplary functions performed by an 
operation of the computer 5 in accordance with the program 80. The functional structure 
lower than the subtraction part 504 is the same as that of Fig. 4 and the functions identical 
25 to those of Fig. 4 are represented by the same reference signs. Fig. 10 is a flowchart 
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showing the operation flow of the computer 5 shown in Fig. 9. 

In the inspection apparatus 1 having the computer 5 of Fig. 9, first, an image of a 
predetermined region of one block on the substrate 9 is picked up by the image pickup 
part 2 and the reference image data 601 is stored in the image memory 531 (Step S21). 
5 Subsequently, the substrate 9 is moved and an image of a predetermined region of another 
block on the substrate 9 is picked up and inspection image data 611 is stored in the image 
memory 531 (Step S22). 

In the range setting part 501, histograms of the reference image and the 
inspection image are obtained and a specified pixel value range for the reference image is 
10 set on the basis of the histogram of the reference image and a specified pixel value range 
for the inspection image is set on the basis of the histogram of the inspection image (Step 
S23). The specified pixel value ranges may be set manually or automatically. 
Automatic settings of the specified pixel value ranges for the inspection image and the 
reference image use the same method (i.e., algorithm). 
15 Next, the LUT generation part 502 generates a reference image LUT 61a on the 

basis of the reference image data 601 (Step S24) and generates an inspection image LUT 
61b on the basis of the inspection image data 611 (Step S25). Then, two generated 
LUTs 61a and 61b are stored in the image conversion part 503. A method of obtaining 
the inspection image LUT 61b is the same as the method of obtaining the reference image 
20 LUT 61a from the reference image and the specified pixel value range for reference 
image (for example, the method shown in Figs. 6 to 8), except that the inspection image 
LUT 61b is generated on the basis of the specified pixel value range for inspection image 
and the histogram of the inspection image. 

In the image conversion part 503, values of pixels in the reference image are 
25 transferred in accordance with the reference image LUT 61a (Step S26) and values of the 
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corresponding pixels in the inspection image are transferred in accordance with the 
inspection image LUT 61b (Step S27). Thus, the converted reference image and the 
converted inspection image can be substantially obtained. 

Both the pixels after transfer are inputted to the subtraction part 504, where a 
5 differential absolute value is obtained (Step SI 7 of Fig. 5), and the differential absolute 
value is compared with a predetermined threshold value by the comparison part 505 (Step 
SI 8) to acquire an inspection result for each pixel. Specifically, the differential absolute 
value image between the converted reference image and the converted inspection image is 
substantially obtained as an enhanced differential image, and further binarized by the 

10 predetermined threshold value. The inspection result, i.e., the binary image indicating 
defective pixels is stored in the memory 532 as the inspection result data 603 and 
displayed on the display 55 if necessary (Step SI 9). 

Thus, in the computer 5 of Fig. 9, the LUTs which are used to convert the images 
are obtained for the reference image and the inspection image. This makes it possible to 

15 reduce an effect due to difference in quality of these images even if the reference image 
and the inspection image have different quality of image (e.g., brightness or distribution 
range of pixel values or the like). 

For example, even if the histograms of the reference image and the inspection 
image each have bi-modality as shown in Fig. 6 and the positions of the peaks are 

20 deviated in these histograms, since each specified pixel value range is set with reference 
to the valley between the peaks, such a conversion as to relatively enhance the difference 
between any two pixel values in the valley in each image is performed. As a result, in 
each of the reference image and the inspection image, the pixel value of the enhanced 
differential image is appropriately enhanced relatively to a pixel which is hard to decide 

25 the specific region (for example, a wiring pattern or a background region on the substrate 



18 

9) it belongs to. 

With the operation of Fig. 10, even if the histogram has one peak, it is possible to 
suppress an effect due to deviation between the position of the peak in the histogram of 
the reference image and that in the histogram of the inspection image. 

5 Fig. 11 is a graph illustrating another example for setting the specified pixel 

value range and the transfer characteristics (LUT) on the basis of the histogram in the 
inspection apparatus 1. The histogram 62a of Fig. 11 has two peaks like that of Fig. 6 
and the pixel value ranges 641 and 642 in two regions on the substrate 9, which 
correspond to these peaks, are obtained. Then, pixel value ranges other than the pixel 

10 value ranges 641 and 642 are set as specified pixel value ranges 63a, 63b and 63c. 

Therefore, a transfer curve 73 which corresponds to the LUT has inclination 
which becomes larger in three specified pixel value ranges 63a, 63b and 63c and becomes 
smaller in the two pixel value ranges 641 and 642. As a result, a value of a pixel in the 
enhanced differential image which corresponds to the pixel belonging to one of the three 

15 specified pixel value ranges in the reference image and (or) the inspection image is 
enhanced. In other words, all the pixels each of which is uncertain on whether it belongs 
to a specific region on the substrate 9 or not are enhanced in the enhanced differential 
image. 

Fig. 12 is a graph illustrating the transfer curve 74 in a case where there are three 
20 (kinds of) regions on the substrate 9 and a histogram 62c having three peaks is obtained. 
For example, as shown in Fig. 13, when a reference image (or inspection image) 700 has 
a background region 701, a first wiring region 702 and a second wiring region 703 and 
pixel values on a line 700a are shown as a graph 700b, the histogram 62c illustrated in Fig. 
12 is obtained. 

25 A method of generating the transfer curve 74 of Fig. 1 2 is the same as that of Fig. 
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11, and the inclination of the transfer curve 74 is small in the pixel value ranges 
corresponding to the regions, pixel value ranges which do not correspond to these regions 
(including an extremely dark portion and an extremely bright portion) is set as the 
specified pixel value ranges, and the inclination of the transfer curve 74 is large in the 
5 specified pixel value ranges. When it is clear that the pixel values of a defect to be 
detected are present only in the specified pixel value range, it is not necessary to set many 
specified pixel value ranges as shown in Fig. 12. 

Fig. 14 is a graph illustrating setting of a specified pixel value range in a case 
where it is known in advance that there are three regions on the substrate 9 and a 
10 histogram 62d does not have peaks corresponding to these three regions. 

When the histogram 62d of Fig. 14 is obtained, first, by using the method 
proposed in the above-mentioned document by Nobuyuki Otsu, two threshold values TH1 
and TH2 for dividing the histogram 62d into three parts are obtained. Then, average 
values AV11, AVI 2 and AVI 3 of the three divided portions in the histogram 62d are 
15 obtained. Further, a range between the average value AV11 and the average value AVI 3 
is set as the specified pixel value range 63 and the transfer curve 74 is obtained. 

In the histogram 62d, the pixel value range 651 from a predetermined pixel value 
to the threshold value TH1, the pixel value range 652 from the threshold value TH1 to the 
threshold value TH2 and a pixel value range 653 from the threshold value TH2 to a 
20 predetermined pixel value are regarded as the specific regions on the substrate 9. When 
the pixel value ranges corresponding to these regions are not clear like in the histogram 
62d, however, there arises a necessity of detecting a defect with emphasis on the pixel 
value range 652. Then, a range from the average value AVI 1 to the average value AV13, 
as a range including the pixel value range 652, is set as the specified pixel value range 63 
25 and the transfer curve 75 having a large inclination in this pixel value range is set. 
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As a method of setting the threshold values in the case where peaks 
corresponding to those regions on the substrate 9 are not clearly present in the histogram, 
any other method may be used. Four or more kinds of regions may be present on the 
substrate 9 and it is not always necessary to know the number of kinds of regions in 
5 advance. 

Fig. 1 5 is a diagram showing still another exemplary functions performed by an 
operation of the computer 5 in accordance with the program 80. In Fig. 15, the 
subtraction part 504 is omitted from the structure of Fig. 4 and with the functional 
structure of Fig. 15, the enhanced differential image is generated without generating the 

10 converted reference image data 602. The functional structure lower than the comparison 
part 505 is the same as that of Fig. 4, and the functions identical to those of Fig. 4 are 
represented by the same reference signs. Fig. 16 is a flowchart showing the operation 
flow of the computer 5 of Fig. 15. 

In the inspection apparatus 1 having the computer 5 of Fig. 15, first, the 

15 reference image data 601 is acquired by the image pickup part 2 and stored in the image 
memory 531 (Step S31), and the range setting part 501 performs setting of the specified 
pixel value range on the basis of the histogram of the reference image (Step S32). 

Next, the LUT generation part 502 generates a 2-D (two-dimensional) LUT 61c 
on the basis of the reference image data 601 (Step S33) and the 2-D LUT 61c is stored in 

20 the image conversion part 503. The 2-D LUT 61c is a table whose inputs are a pixel 
value of the reference image and a value of the corresponding pixel of the inspection 
image, indicating the transfer characteristics for determining a pixel value of an enhanced 
differential image corresponding to the two inputs. Then, in the inspection apparatus 1, 
when the image pickup part 2 acquires the inspection image data (Step S34), the image 

25 conversion part 503 obtains each of pixel values of the enhanced differential image (Step 
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S35) and the comparison part 505 compares the pixel value of the enhanced differential 
image with the threshold value, to thereby perform an inspection (Step S36). A 
comparison result is displayed on the display 55 if necessary (Step S3 7). 

Figs. 17A and 17B are views showing characteristics of an example of the 2-D 
5 LUT 61c generated by the LUT generation part 502, Fig. 17A shows a transfer curve 
(transfer characteristics) 711 in a case where it is supposed that the converted reference 
image (and the converted inspection image) should be generated, like in Fig. 4, and Fig. 
17B schematically shows a state of the 2-D LUT 61c for generating the enhanced 
differential image in accordance with the transfer curve 711 of Fig. 17 A. In Fig. 17B, 

10 the lower left is the origin (0, 0), and the horizontal axis indicates the pixel values of the 
inspection image and the vertical axis indicates the pixel values of the reference image. 
Then, a value of the table in a coordinate determined by these pixel values is specified as 
a pixel value of the enhanced differential image. The broken lines of Fig. 17B indicate 
magnitude of values included in the 2-D LUT 61c as contour lines. 

15 Figs. 18A and 18B are views showing characteristics of a 2-D LUT illustrated 

for reference. Fig. 18A shows a transfer curve 712 without transfer of pixel values (i.e., 
straight line), and Fig. 18B schematically shows the 2-D LUT corresponding to the 
transfer curve 712 of Fig. 18 A. In the 2-D LUT of Fig. 18B, values in the 2-D LUT are 
in proportion to the distance from a diagonal line 81 so that the difference between the 

20 pixel value of the inspection image and the pixel value of the reference image should be 
an output value. 

On the other hand, in the 2-D LUT 61c of Fig. 17B, a value "0" is stored on the 
diagonal line 81 but values in coordinates are set so that the contour lines protrude 
towards the center. As a result, when both the pixel value of the inspection image and 
25 the pixel value of the reference image belong to a pixel value range of the Fig. 17A which 
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has a sharp inclination (i.e., in the neighborhood of the center of Fig. 17B), a relatively 
larger value than the difference between the pixel value of the inspection image and the 
pixel value of the reference image is specified in accordance with the 2-D LUT 61c. In 
contrast to this, when both the pixel value of the inspection image and the pixel value of 
5 the reference image belong to a pixel value range of the Fig. 17A which has a gentle 
inclination (i.e., on a side of the origin or the opposite side of the origin in Fig. 17A), a 
relatively smaller value than the difference between the pixel value of the inspection 
image and the pixel value of the reference image is specified in accordance with the 2-D 
LUT 61c. 

10 Thus, the 2-D LUT 61c generated by the LUT generation part 502 is a table for 

obtaining the pixel value of the enhanced differential image directly and efficiently from 
the pixel value of the inspection image and the pixel value of the reference image without 
conversion of the reference image and the inspection image in accordance with the 
transfer characteristics. It is thereby possible to perform inspection in the inspection 

1 5 apparatus 1 at a high speed. 

Fig. 19 is a diagram showing another exemplary functional structure of the 
computer 5 in a case where the same operation as that for generating the enhanced 
differential image by using the reference image LUT 61a and the inspection image LUT 
61b of Fig. 9 is performed by using the 2-D LUT. The functional structure of Fig. 19 is 

20 different from that of Fig. 9 in only that the subtraction part 504 is omitted and a 2-D 
LUT 61d is provided in the image conversion part 503. 

In an operation of the computer 5 of Fig. 19, the same operations as those of 
Steps S21 to S23 of Fig. 10 are first performed and after that, the same operations as those 
of Steps S33 and S35 to S37 of Fig. 16 are performed. Specifically, the reference image 

25 data 601 and the inspection image data 611 are prepared in the image memory 531 (Steps 
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S21 and S22) and the range setting part 501 sets the specified pixel value ranges on the 
basis of the histograms of these pixel values (Step S23). Then, the LUT generation part 
502 generates the 2-D LUT 61 d on the basis of the specified pixel value ranges (Step 
S33). 

5 Figs. 20A and 20B are views showing characteristics of an example of the 2-D 

LUT 6 Id. In Fig. 20A, a transfer curve 713 shows transfer characteristics derived from 
the reference image and a transfer curve 714 shows transfer characteristics derived from 
the inspection image. Correspondingly to these transfer characteristics, the 2-D LUT 
61d of Fig. 20B are generated by the LUT generation part 502. The LUT generation 
10 part 502 may directly generate the 2-D LUT 61d from the specified pixel value ranges set 
for the reference image and the inspection image without obtaining the transfer curves of 
Fig. 20A. 

In the 2-D LUT 61d of Fig. 20B, a value "0" is set approximately on a curve 81a 
and the contour lines are distorted as compared with those of the 2-D LUT 61c of Fig. 

15 17B. This makes it possible to generate the enhanced differential image while reducing 
an effect due to difference in quality of these images even if the reference image and the 
inspection image have different quality of image (e.g., brightness or distribution range of 
pixel values), and therefore the same inspection as that by the structure of Fig. 9 can be 
performed at a high speed (see Steps S3 5 to 37). 

20 Though appropriate inspection in accordance with the nature of defect is 

performed by obtaining the enhanced differential image in the preferred embodiment 
discussed above, the enhanced differential image may be processed by using the 
differential absolute value image (hereinafter, referred to as a "simple differential image") 
between the original reference image and the original inspection image. Fig. 21 is a 

25 diagram showing part of functional structure of the computer 5 in a case where the 
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enhanced differential image and the simple differential image are synthesized in the 
structure of Fig. 4 or 9. In Fig. 21, the subtraction part 504 of the structure of Fig. 4 or 9 
is replaced by a first subtraction part 504a, a second subtraction part 504b and a 
synthesizing part 507. Other constituent elements are identical to those of Fig. 4 or 9, 
5 and the elements are represented by the same reference signs in the following description. 

Though Fig. 21 shows that the reference image data 601, the converted reference 
image data 602, the inspection image data 611 and the converted inspection image data 
612 are stored in the image memory 531, there may be a case where the data other than 
the reference image data 601 are not stored in the image memory 531 but are inputted 

10 directly to the first subtraction part 504a and the second subtraction part 504b. 

When inspection is performed by the structure of Fig. 21, an operation of Fig. 22 
is performed instead of Step SI 8 of Fig. 5. Also in the structure of Fig, 21, the LUT 61 
of Fig. 4, the reference image LUT 61a and the inspection image LUT 61b of Fig. 9 are 
generated, the reference image data 601 and the inspection image data 611 are converted 

15 by the image conversion part 503 and the converted reference image data 602 and the 
converted inspection image data 612 are obtained (Steps Sll to SI 6 of Fig. 5, or Steps 
S21 to S27 of Fig. 10). 

After that, these converted image data are inputted to the second subtraction part 
504b and data of the enhanced differential image is generated (Step S17 of Fig. 5). At 

20 that time, the reference image data 601 and the inspection image data 611 are inputted to 
the first subtraction part 504a and data of the simple differential image is obtained (Step 
SI 81 of Fig. 22). Each pixel value of the enhanced differential image and the value of 
the corresponding pixel of the simple differential image are inputted to the synthesizing 
part 507 and the square root of (multiplication) product of these pixel values is obtained 

25 as a synthesized pixel value (Step SI 82). With this operation, data of a synthesized 
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differential image which is obtained by substantially synthesizing the enhanced 
differential image and the simple differential image is generated. 

Each of the pixel values in the synthesized differential image is compared with a 
predetermined threshold value by the comparison part 505 (Step SI 83), and the 
5 comparison result is stored in the memory 532 as the inspection result data 603 (see Fig. 
4) and displayed on the display 55 if necessary (Step SI 9 of Fig. 5). 

Thus, in the computer 5 comprising the structure of Fig. 21, detection of 
defective pixels is performed by synthesizing the enhanced differential image and the 
simple differential image and comparing the synthesized image with the predetermined 

10 threshold value. When a transfer curve(s) indicating the characteristics for converting 
the images is set through computation by the computer, it is preferable that the transfer 
curve should be bent as shown in Figs. 6 to 8. In some cases, however, the enhancement 
of differential image is excessively performed by the bent line. Then, in such a case, the 
degree of enhancement is relieved by synthesizing the enhanced differential image and 

15 the simple differential image in the structure of Fig. 21, and appropriate inspection is 
thereby achieved. 

Fig. 23 is a diagram showing a structure for synthesizing the enhanced 
differential image and the simple differential image in a case of using the 2-D LUT (see 
Fig. 15 or 19). In the structure of Fig. 23, a 2-D LUT is provided in the image 

20 conversion part 503, and an enhanced differential image is directly generated by the 
image conversion part 503 from the reference image data 601 and the inspection image 
data 611 and inputted to the synthesizing part 507. On the other hand, subtraction of the 
reference image data 601 and the inspection image data 611 is performed by the 
subtraction part 504 to generate data of a simple differential image and the data of simple 

25 differential image is inputted to the synthesizing part 507. Then, the enhanced 
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differential image and the simple differential image are synthesized by the synthesizing 
part 507 and each pixel value of the synthesized differential image is compared with a 
predetermined threshold value by the comparison part 505, to perform an inspection. In 
other words, the same operation as that in the structure of Fig. 21 is performed, except 
5 that the data of enhanced differential image is directly generated by the image conversion 
part 503. 

In the 2-D LUT of the image conversion part 503, values including the 
synthesizing operation may be stored. In this case, the structure of the inspection 
apparatus 1 is the same as that of Fig. 15 or 19. Fig. 24 is a view showing a 2-D LUT 
10 which is obtained by adding the synthesizing operation to the 2-D LUT of Fig. 17B. 
Figs, 25A to 25D are graphs showing a relation between the pixel values of the reference 
image and values stored in the LUT (i.e., the pixel values of the synthesized differential 
image) in a case where the pixel values of the inspection image are 50, 100, 125 and 150, 
respectively. 

15 As shown in Fig. 24, the 2-D LUT has a gentle change (i.e., distribution) of 

values in the LUT as compared with that of Fig. 17B. In Figs. 25 A to 25D, the solid line 
indicates a relation between the pixel values of the reference image and the values in the 
LUT of Fig. 24, the broken line indicates a relation between the pixel values of the 
reference image and the values in the LUT in Fig. 17B and the alternate long and short 

20 dash line indicates a relation between the pixel values of the reference image and the 
values in the LUT in Fig. 1 8B (in other words, the LUT in a case where enhancement of 
the differential absolute value image is not performed). 

As shown in figs, 25A to 25D, in an image converted in accordance with the 2-D 
LUT of Fig. 24, the degree of enhancement of the enhanced differential image is relieved 

25 with the simple differential image. By directly generating the synthesized differential 
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image with the 2-D LUT, it is possible to simplify the computation of the computer 5 in 
the inspection apparatus 1 . 

In the inspection apparatus 1, though the whole reference image and the whole 
inspection image are compared in the above discussion, the inspection may be performed 
5 for part of the reference image and part of the inspection image. In other words, part of 
the image acquired by the image pickup part 2 may serve as the inspection image in the 
above discussion. Hereinafter, the image acquired by the image pickup part 2 is referred 
to as a whole inspection image and part of the whole inspection image to be computed is 
referred to as a partial inspection image. In accordance with this, the whole and part of 

10 the reference image are also referred to as a whole reference image and a partial reference 
image, respectively. Fig. 26 is a flowchart showing an operation flow of the inspection 
apparatus 1 in a case where inspection is performed partial inspection image. 

First, when data of the whole reference image and the whole inspection image 
are acquired by the image pickup part 2 and prepared (Step S41), the whole reference 

15 image and the whole inspection image are divided into a plurality of partial reference 
images and partial inspection images (Step S42). Fig. 27 is a view illustrating division 
of an image 660 (the whole reference image or the whole inspection image) into a 
plurality of partial images 661 . The above operation is performed for data of one partial 
reference image and data of one partial inspection image, an enhanced differential image 

20 or a synthesized differential image corresponding to the partial inspection image is 
obtained and each pixel in the enhanced differential image or the synthesized differential 
image is compared with a threshold value, to perform inspection (Step S43). 

When the inspection for one partial inspection image is finished, inspection for 
the next partial inspection image is performed and when the inspection for all the partial 

25 inspection images are completed, the inspection for the whole inspection image is thereby 
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finished (Step S44). By performing the inspection after dividing the whole inspection 
image (and the whole reference image), even if regions in the whole inspection image are 
different in quality of image from one another, appropriate inspection can be performed 
on the whole of inspection image. 
5 Though the preferred embodiment of the present invention has been discussed 

above, the present invention is not limited to the above-discussed preferred embodiment, 
but allows various variations. 

For example, the inspection image and the reference image may be part of an 
image acquired by the image pickup part 2. There may be a case, for example, where an 

10 image having repeating patterns is acquired by the image pickup part 2 and part of the 
image serves as a reference image and other part serves as an inspection image. Further, 
the reference image may be an average image of a plurality of images on the substrate 9 
which correspond to one another or the reference image data may be generated in advance 
on the basis of a design data of pattern on the substrate 9. 

1 5 Though the LUT(s) is generated on the basis of only the reference image or on 

the basis of the reference image and the inspection image in the above-discussed preferred 
embodiment, the LUT may be generated on the basis of only the pixel values of the 
inspection image. By obtaining the LUT on the basis of the reference image or the 
inspection image, the characteristics of the reference image or the inspection image can 

20 be reflected in the LUT and automatic generation of LUT can be achieved. 

The transfer curve indicating the transfer characteristics in accordance with the 
LUT is not needed to be bent (i.e., lines) but may be a smooth curve only if such 
conversion as to enhance the difference between any two pixel values in the specified 
pixel value range can be performed. 

25 Though the enhanced differential image is generated so that the difference of any 
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(two) pixel values should be enhanced within the specified pixel value range which is 
specified in the defect detection in the above-discussed preferred embodiment, the 
specified pixel values may be discrete. In other words, there may be a case where only 
the difference between any pixel values among a plurality of specified pixel values is 
5 enhanced. This allows appropriate inspection with attention to the specified pixel values. 
The specified pixel value range in the above preferred embodiment can be regarded as a 
set of specified pixel values or a range including a plurality of specified pixel values. 

It is not necessary that the whole range of pixel values of the converted image 
should be equal to the whole range of pixel values of the image before conversion in the 

10 structures of Figs. 4, 9 and 21. There may be a case, for example, where the range of 
pixel values before conversion is from 0 to 255 and the range of pixel values after 
conversion is from 0 to 1 which are high-precision real numbers. Similarly, the range of 
pixel values in the enhanced differential image obtained by the image conversion part 503 
in the structures of Figs. 15, 19 and 23 may be from 0 to 1 which are high-precision real 

15 numbers. The threshold value which is compared with the enhanced differential image 
or the synthesized differential image is set as appropriate in accordance with the range of 
pixel values of the image. After transfer to the pixel values ranging from 0 to 1, the 
pixel values may be further transferred to those in the range from 0 to 255. 

In Figs. 21 and 23, as a method of synthesizing the enhanced differential image 

20 and the simple differential image, other methods may be used. A (multiplication) 
product of the pixel value of the enhanced differential image and the pixel value of the 
simple differential image may become an object to be compared in the comparison part 
505, and synthesis may be performed by adding the pixel value of the enhanced 
differential image and the pixel value of the simple differential image at a constant ratio. 

25 As a specific example, the pixel value of the synthesized differential image may be 
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obtained by synthesizing the pixel value of the enhanced differential image and the pixel 
value of the simple differential image with parameter t (0 < t < 1) (this parameter may be 
(O^St^l)) at the ratio of (t : (1-t)). In this case, by changing the parameter t, it is 
possible to easily change the degree of synthesis of the enhanced differential image and 
5 the simple differential image. 

It is not necessary to perform the inspection on each pixel, and for example, a set 
of pixels (e.g., 2X2 pixels) may be handled as one pixel in the above preferred 
embodiment. In other words, the pixel in the above preferred embodiment is not needed 
to strictly correspond to a physical pixel which is a constituent of an image. 

10 Though the image acquired by the image pickup part 2 is processed by the 

computer 5 in the above-discussed preferred embodiment, part of the functions shown in 
Figs. 4, 9, 15, 19, 21 and 23 may use dedicated circuits for faster operation. Especially, 
the functions of the image conversion part 503, the subtraction part 504, the comparison 
part 505 and the like can be easily performed by dedicated circuits, and using dedicated 

15 circuits allows faster inspection after generation of the LUT(s). 

The inspection apparatus 1 can be used not only for the inspection of pattern on a 
semiconductor substrate but also for inspection of fine patterns on various substrates, such 
as patterns formed on a color filter, a shadow mask, a high-definition printed circuit board 
or the like, pattern of lead frame, pattern of photomask used for forming these patterns or 

20 the like. 

While the invention has been shown and described in detail, the foregoing 
description is in all aspects illustrative and not restrictive. It is therefore understood that 
numerous modifications and variations can be devised without departing from the scope 
of the invention. 
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